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In the Claims 

1 . (original) A method for recovering non-rigid 3D shape and motion of an object 
from a video of the object, comprising: 

determining correspondences of a set of features on the object in each image 
of the video; 

factoring the correspondences into a motion matrix and a shape matrix; and 
extracting a 3D model from the factored motion matrix and the shape matrix 
that conforms to the object in the video, the 3D model including a linear basis for 
deformable shape of the object in the video, and for each image a 3D rotations 
matrix, deformation coefficients, and translation vectors. 

2. (original) The method of claim 1 wherein the 3D model has a minimum residual 
error and minimal deformation coefficients. 

3. (original) The method of claim 1 further comprising: 

identifying the set of features in a reference frame of the video; 

determining a location for each feature in the reference frame; 

determining a corresponding location of each feature in each other frame of 
the video, the location of each feature in the reference frame and the locations of the 
corresponding features in each other frame forming the correspondences. 

4. (original) The method of claim 1 further comprising: 

selecting a first image of the video as the reference frame. 
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5. (original) The method of claim 1 further comprising: 

selecting a middle image of the video as the reference frame. 

6. (original) The method of claim 1 further comprising: 

determining a location of each feature in each frame of the video, and 
gradients of image intensities in regions around each feature by comparing to the 
corresponding region in the reference frame. 

7. (original) The method of claim 1 wherein each region is approximately 5x5 
pixels centered on each feature. 

8. (currently amended) The method of claim 5 claim 6 wherein the correspondences 
between successive images of the video are constrained to be consistent with a 
range of possible projections of anon-rigid 3D surface of the object having a finite 
number of deformations to enable the determining of the correspondences when 
regions within the images lack identifying marks and texture. 

9. (original) The method of claim 1 wherein the correspondence include 
quantification of spatial and temporal variations of image intensities around each 
feature, the quantification determined from gradients of the image intensities. 

10. (currently amended) The method of claim 8 claim 9 wherein the spatial 
variation at any location is X, the temporal variation between that location and a 
corresponding location in another image is Y, and X is an estimate of a covariance 
of an uncertainty in the temporal variations Y. 
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1 1 . (currently amended) The method of claim 9 claim 10 wherein the factoring uses 
a covariance of the spatial variations X to decompose the temporal variations Y into 
low-dimensional factors of the correspondences that preserve information about the 
temporal variations Y in a finite number of dimensions, and a distribution of the 

w. uncertainty in the low-dimensional factors is identically independently Gaussian, 
regardless of correlational structure of noise and error in the correspondences. 

12. (original) The method of claim 1 further comprising: 

generating a novel video from the video by changing any combination of the 
shape matrix, the 3D rotations matrix, the deformation coefficients, and the 
translation vectors of the 3D model. 

13. (currently amended) The method of claim 6 claim 7 wherein the constraints of 
the factoring enforces the range of possible projections of the non-rigid 3D surface 
having the finite number of deformations to produce the motion matrix and the 
shape matrix. 

14. (currently amended) The method of claim 12 claim 13 further comprising: 

transforming the motion matrix and the shape matrix to be consistent with 
image formation governing 2D projections of the non-rigid 3D object. 

15. (original) A method for recovering a 3D model of non-rigid 3D shape and 
motion of an object directly from an input video of the object, comprising: 

identifying a set of features on the object in a reference image of the input 

video; 

determining correspondences between the set of features in the reference 
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image and corresponding features in each other image of the input video; 

factoring the correspondences into a motion matrix and a shape matrix; 

extracting the 3D model from the factored motion matrix and the shape 
matrix, the 3D model including a linear basis for deformable shape of the object in 
the input video, and for each image a 3D rotations matrix, deformation coefficients, 
and translation vectors; and 

manipulating the 3D model and the input video to generate a novel video 
from the input video. 

16. (original) A system for recovering a non-rigid 3D model of a scene, 

a camera configured to acquire an input video of the scene; 
means for identifying a set of features in the scene in a reference image of the 
input video; 

means for determining correspondences between the set of features in the 
reference image and corresponding features in each other image of the input video; 

means for factoring the correspondences into a motion matrix and a shape 
matrix; 

means for extracting the non-rigid 3D model from the factored motion matrix 
and the shape matrix, the 3D model including a linear basis for deformable shape in 
the scene, and for each image a 3D rotations matrix, deformation coefficients, and 
translation vectors. 

17. (currently amended) The system of claim 15 claim 16 further comprising: 

a display device configured to display a novel video generated by 
manipulating the non-rigid 3D model and the input video. 
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18. (currently amended) The system of claim 15 claim 16 wherein the scene 
includes a non-rigid deformable object. 

19. (currently amended) The system of claim 15 claim 16 further comprising: 

a pointing device 106 to identify the set of features in the reference image. 

20. (currently amended) The system of claim 15 claim 16 further comprising: 

a computerized vision system to automatically identify the set of features in 
the reference image. 
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